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Abstract 

Our interaction with digital resources is becoming increasingly based on touch, gestures, and 
now also eye movement. Many everyday consumer electronics products already include 
touch-based interfaces, from e-book readers to tablets, and from the last personal computers to 
the GPS system in your car. What implications do these new forms of interaction offer for 
language learning and teaching? In this short article, I will first consider some of the most 
recent technological developments for their pedagogic potential, and in particular for their 
support for embodied and extended cognition. Next, I will offer some suggestions for 
researching their impact on learning and teaching, as well as ways in which teachers and 
materials developers can capitalise on these advances in technology to support more 
interactive and dynamic forms of learning. 
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1. Touch and gesture-based technologies 

Advances in our understanding of the ways people interact with information, their natural 
responses to new environments to locate and retrieve that information, and their ways of 
sharing it with others, are examples of insights that are starting to challenge our conceptions 
of the ways people learn, and by extension how we teach. Extended cognition (Atkinson 
2010), for example, emphasises the inextricable connection between the mind and the 
environment and embodied cognition “views cognitive activity as grounded in bodily states 
and activities” (Atkinson, 2010, p. 599). What both of these conceptions of cognition have in 
common is the role of the physical world in our thinking, and by extension, our learning. For 
example, spontaneous gestures have been shown to support thinking and learning, and there is 
also evidence that designed gestures, as well as manipulation of objects (e.g. on a screen) can 
have an impact on learning (Segal, 2011). 
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Recently, new ways of interacting with information have become widely available as a 
result of a range of technological developments that make more use of touch, gestures and 
other bodily actions. For example, touch is now an important feature of many devices. Button 
or screen presses are now commonly acknowledged with so-called ‘haptic feedback’ (a brief 
vibration to indicate your command has been received). In addition, multiple-touch 
commands are now commonly used to manipulate phones, tablets, and other devices. Swiping 
to move objects across a screen, ‘pinching’ to zoom out or out, and more complex multiple- 
finger touch combinations are examples of fairly recent ways of interacting with devices. (It is 
hard to imagine that only a few years ago this type of interaction did not exist in mainstream 
electronics). Recent desktop computers and laptops increasingly have touch built-in as an 
additional way of manipulating content. 

Motion is another recent development allowing users to move their phone in a 
particular direction to us it as a compass, or to turn the phone upside down to turn the phone 
off. In-built accelerometers recognise when users tilt their phone, and let them move objects 
on the screen, increase or decrease the volume, and so on. 

Gestures and movements are also becoming increasingly common. Microsoft’s Kinect 
technology, for instance, recognises game players’ movements, allowing them to use their 
whole bodies to control game characters, objects and the like. Gesture control is also being 
built into mobile devices, letting users wave across the screen to turn sound on or off, for 
example, and into webcams. More recent developments, such as Leap Motion’s gesture 
recognition system, will recognise individual finger movements and can be used to type or 
draw simply by holding your hand in the air in front of a screen. 

Eye-tracking research has given us considerable insight into cognitive processes, and 
has also informed research in language learning (for example, to show which aspects of the 
input learners are paying attention to). Although still in its infancy, developments in eye 
movement are showing considerable promise in allowing users to manipulate objects on the 
screen simply by focusing their eyes on a particular point, such as Tobii’s Gaze technology 
(http://www.tobii.com) . 

2. Research on touch-based technologies in education 

The brief examples above are related to everyday object manipulation or games, but what 
impact do these types of interaction have on education? To some extent, this is still uncharted 
territory. Apart from the use of, for example, eye-tracking for the purposes of research, the 
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most common application of this technology for learning has been incorporating touch as a 
response mechanism in (educational) games. Embodied and extended cognition theory 
(Atkinson, 2010) argues that cognition is inherently connected to bodily sensations and that 
interaction with the environment plays a key role in learning, even in supposedly abstract 
cognitive tasks. Dijkstra, Kashak & Zwaan (2007), for example, have shown that 
verbalization of memory is facilitated when learning assumes original body posture during 
recall. Embodied interaction (or the relationship between the purely biological and the 
cognitive) is a type of sociocognitive action that touch and gestural interfaces have the 
potential to support. 

Kuznetsov (2009) employed wearable bracelets that mapped distinct haptic pulse 
signatures to new concepts. When these concepts were re-encountered at a later time, the 
corresponding haptic cues were replayed to aid recall and recognition. The authors found that 
the feedback helped in particular those with impaired memory and that the bracelets worked 
discreetly in comparison with existing technology that relies on audio or video cues that may 
be disruptive in daily use. Other studies have investigated the use of haptic feedback in the 
areas of flight simulation (Sankaranarayanan et al., 2003), dentistry and high school physics 
classes (Hamza-Lup & Adams, 2008). 

3. Ideas for possible use of touch-based technologies in the language classroom 

Beyond interacting with devices on which language content is stored, very little use has been 
made of touch in language education. Some ideas for how this could be done are presented 
below. 

Some language teaching methods specifically draw on tactile activities to support 
learning. For example, Total Physical Response (Asher, 1977) is based on the premise that a 
combination of a visual and/or auditory with a tactile (touch) and kinaesthetic (body 
posture/movement) experience is more likely to be incorporated into the learner’s developing 
language system than exposure to input through visual or oral means alone. In TPR, learners 
respond to commands from the teacher by getting up, walking around, touching or pointing at 
objects, or through other movements. 

Although TPR was never widely used, it did help to increase interest in the role of 
kinaesthetic experiences in learning languages. In the late 1970s an interest developed in the 
ways in which different learners acquired new knowledge. This focus on ‘learning styles’ 
emphasised the importance of direct, physical experience for learners (Naiman et al 1978). 
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Many of the motion-based technologies above could be employed to include a kinaesthetic 
experience in the language learning process; for example, in my own classes I have used an 
XBOX with a Kinect sensor to teach beginner level students different action verbs. 

Motion technologies can also help expand the range of exercise types for students. For 
example, information can be responded to by moving an object on a phone screen (e.g. to re¬ 
order the different words in a sentence) or by pointing the device in a particular direction (for 
example to ‘point to’ the correct answer to a question). This can be particularly helpful for 
self-study materials to offer a larger number of options for interacting with language content. 
For example, one application is for the teaching of directions or other spatial directives. 
Questions can require learners to move objects across the screen or to point their phones in a 
particular direction. Quizzes like this can be created using free software such as one of the 
many available app creators. 

A more ambitious project would allow learners to construct or deconstruct sentences 
by clause, or collocation, verbal phrase, or other segmentations. Being able to visually see the 
building blocks of the language while being able to manipulate them could make language 
associations stronger and more durable. Zooming in or out from text to paragraph, and from 
sentence to word and to phoneme level, could be helpful for teachers and learners alike. 

Particularly promising is the effect of the new technologies described above on the 
ways in which learners can be supported through different types of feedback. Frequent 
feedback has been shown to have a significant impact on learning, but feedback given during 
task completion can be disruptive, whether given by a teacher through oral comments or by a 
computer through on-screen visuals. Haptic feedback, or a brief vibration from the device 
being used, is one way to indicate, for example, that a response was incorrect. Or, more 
positively, haptic feedback could be given whenever a learner gets an answer right. 

Another use for haptic feedback is to indicate sentence or word stress. By combining 
oral and haptic information, learners may find it easier to identify common stress patterns, for 
example. Hwang & Cho (2012) used this to teach intonation. Related to this is the use of 
haptic feedback as a form of input enhancement (attempts to direct the learner’s attention to a 
specific linguistic form in the target language input; Sharwood Smith, 1993) 

Touch and motion are common features in many games and play an important role in 
allowing players to control their environment and in giving them feedback. I have written 
about games and their benefits for language learning before (Reinders 2010, 2012), but in a 
nutshell, games have been shown to be successful in engaging students more deeply into the 
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learning process. A great way of breaking normal classroom routine is to use an XBOX with a 
Kinect sensor. One way of using this is to choose games that include a lot of language. 
Adventure games include many instructions and imperatives (‘Jump!’, ‘Go to the temple as 
fast as you can’) and your learners need to respond by moving around. Keep the activities 
short so that all students get a chance. There is no need for only one or two players respond, 
though - make room in the class and everyone can play along (but of course only the players’ 
actions are recorded by the sensor). Of course, not all schools have an XBOX lying around, 
but maybe you or one of your colleagues (or more likely, their kids!) have one at home that 
you could boiTow for a day. 

There is also potential for gestures technologies to support the acquisition of non¬ 
verbal communication. Most languages use gestures to support meaning, and some languages 
famously frequently so, such as Italian. Applications that recognise the gestures students 
make linked with the meaning they are orally trying to convey could help with the acquisition 
of this important aspect of the language. 

Although beyond the scope of this article, another obvious application is for the 
teaching of sign language. A number of mobile applications now exist that start to make use 
of touch and motion. 

4. Where do I start? 

If you decide to attempt any of the ideas above, and as with all teaching activities, you need to 
consider what you are trying to achieve. Playing XBOX games may work really well with 
beginner level school children, but not go down so well with executives on a Business English 
course. For the latter group, flashcards with haptic feedback that they practise with on their 
mobile phones between meetings may be more suitable - and will give them immediate 
feedback. Of course, there is also the question of what is the most effective. Computer games 
may be entertaining but if they don’t contribute to learning, they are obviously not suitable for 
use in class. 

Once you have decided on your outcome, think of the activity that your students will 
complete. In the TPR example above, what will be the language focus? Will it be on 
following directions? Or on responding to spatial commands (‘Move the book to the left’)? 
You will also have to consider how to connect the activity with the preceding and following 
lesson content. For example, learners can be asked to use the commands they heard when 
playing a game in their subsequent group work?. 
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It is important to realise, though, that many of these technologies are in their infancy; 
there are no dedicated materials, set guidelines, or examples of best practice to follow. In this 
sense, the use of motion technologies is a form of language teaching experimentation. Expect 
for some things to go wrong, but to also leam a lot during the process. 
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